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Abstract. We register a random sequence which has the following 
properties: it has three segments being the homogeneous Markov processes. 
Each segment has his own one step transition probability law and the length 
of the segment is unknown and random. It means that at two random mo- 
ments 01,02, where ^ 0i ^02, the source of observations is changed and 
the first observation in new segment is chosen according to new transition 
probability starting from the last state of the previous segment. In effect the 
number of homogeneous segments is random. The transition probabilities 
of each process are known and a priori distribution of the disorder moments 
is given. The fonner research on such problem has been devoted to vari- 
ous questions concerning the distribution changes. The random number of 
distributional segments creates new problems in solutions with relation to 
analysis of the model with deterministic number of segments. Two cases 
are presented in details. In the first one the objectives is to stop on or be- 
tween the disorder moments while in the second one our objective is to find 
the strategy which immediately detects the distribution changes. Both prob- 
lems are reformulated to optimal stopping of the observed sequences. The 
detailed analysis of the problem is presented to show the form of optimal 
decision function. 
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1. INTRODUCTION 



Suppose that the process X = {Xn,n G N}, N = {0, 1,2,...}, is observed sequen- 
tially. The process is obtained from three Markov processes by switching between them at 
two random moments of time, 6i and 62. Our objective is to detect these moments based on 
observation of X. 

Such model of data appears in many practical problems of the quality control (see 
Brodsky and Darkhovsky [SI], Shewhart jlTII and in the collection of the papers |12|]), traffic 
anomalies in networks (in papers by Dube and Mazumdar |^, Tartakovsky et al. |22]), 
epidemiology models (see Baron |[lj]). In management of manufacture it happens that the 
plants which produce some details changes their parameters. It makes that the details change 
their quality. Production can be divided into three sorts. Assuming that at the beginning of 
production process the quality is highest, from some moment 61 the products should be 
classified to lower sort and beginning with the moment 62 the details should be categorized 
as having the lowest quality. The aim is to recognize the moments of these changes. 



Shiryaev llS 



n 



a, UM solved the disorder problem of the independent random variables 
with one disorder where the mean distance between disorder time and the moment of its 
detection was minimized. The probability maximizing approach to the problem was used 
by Bojdecki |t3i] and the stopping time which is in a given neighborhood of the moment of 
disorder with maximal probability was found. The disorders in more complicated depen- 
dence structures of switched sequences are subject of investigation by Pelkowitz j 1 4l. 1 1 5n . 
Yakir |24], Mustakides lU], Lai fl [3], Fuh |7], Tartakovsky and Veeravalh lEJ. The 
probability maximizing approach to such problems with two disorders was considered by 
Yoshida Q, Szajowski and Sarnowski and Szajowski Yoshida 11251] investi- 

gated the problem of optimal stopping the observation of the process X so as to maximize 
the probability that the distance between the moments of disorder 9i and their estimates, 
the stopping times Ti, i = 1,2, will not exceed given numbers (for each disorder inde- 



pendently). This question has been reformulated by Szajowski 1211] to the simultaneous 
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detection of both disorders under requirement that the performance of procedure is globally 
measured for both detections and it has been extended to the case with unknown distribu- 



tion between disorders by Sarnowski and Szajowski jlql (see also papers by Bojdecki and 
Hosza 141] for related approach with switching sequences of independent random variables). 
The method of solution is based on a transformation of the model to the double optimal stop- 



ping problem for markovian function of some statistics (see Haggstrom ||8|], Nrkolaev ||12|1 ). 
The strategy which stops the process between the first and the second disorder with maximal 
probability has been constructed by Szajowski 12011 . The considerations are inspired by the 
problem regarding how we can protect ourselves against a second fault in a technological 
system after the occun^ence of an initial fault or by the problem of detection the beginning 
and the end of an epidemic. 

The paper is devoted to a generalization of the double disorder problem considered 
both in 12011 and 12 IH in which immediate switch from the first preliminary distribution to 
the third one is possible (i.e. it is possible that the random variables 9i and 62 are equal with 
a positive probability). It is also possible that we observe the homogeneous data without 
disorder when both disorder moments are equal to 0. The extension leads to serious diffi- 
culties in the construction of an equivalent double optimal stopping model. The formulation 
of the problem can be found in Section |2] The main results are subject of Sections |4] (see 
Theorem gD and|5l 



2. FORMULATION OF DETECTION PROBLEMS 

Let {Xn)n£n be an observable sequence of random variables defined on the space 
(il,jr, P) with values in (^,B), where E is a Borel subset of R. On (E, S) there is a- 
additive measure /i. On the same probability space there ai^e defined random variables 9i, 
62 with values in N and the following distributions: 

(2.1) P{ei=j) = l{j=o}{j)7^ + l{j>o}{jWi~'qi, 

(2.2) P{e2 = k\ei=j) = I{k=j}{k)p + l{k>j}{k)ppt'~\2 
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where j = 0, 1, 2, k = j,j + l,j + 2, n = 1 — vr, p = 1 — p. Additionally we 
consider Markov processes (X^, P^) on (f], JF, P), i = 0, 1, 2, where cr-fields C/,^ are 
the smallest cr-fields for which {X^)'^^q, i = 0,1,2, are adapted, respectively. Let us define 
process {Xn)n£N in the following way: 

We make inference on 9i and 62 from the observable sequence n € N) only. It should 
be emphasized that the sequence (Xn, n € N) is not markovian under admitted assumption 



as it has been mentioned in ll20n . 12411 an d jal . However, the sequence satisfies the Markov 



property given 9i and 62 (see Szajowski 



2l|] and Moustakides [11]). Thus for further con 



sideration we define filtration {Tn}neN, where J^n = <7{Xo,Xi, related to real 

observation. Variables 9i, 62 are not stopping times with respect to Tn and fi-fields Q'. 
Moreover, we have knowledge about the distribution of (^1, 62) independent of any obser- 
vation of the sequence {Xn)neN- This distribution, called the a priori distribution of {9i, ^2) 
is given by (llTT ) and (l2!2l ). 

It is assumed that the measures P^(-) on ^, i = 0, 1, 2, have following representation. 
For any B £ B we have 

Piiu ■.xiGB) = p{xi € B\x^o = ^) = I fi{y)Kdy) = J Plidy) = pUb), 

B B 

where the functions /^(•) are different and fx{y)/fx^^^^°'^^{y) < 00 for i = 0, 1, 2 and all 
x,y gK. We assume that the measures /i^, x G E are known in advance. 

For any Dn = {u : Xi £ Bi, i = 1, . . . , n), where Bi G B, and any x G E define 

Px{Dn) = P{Dn\XQ = x) = j Sn{x,yn)p{dyn) = J Px{dyn) = Px{^i=lBi) , 

X" ,Bi X" ,Bi 

where the sequence of functions Sn '■ x"=iIE ^ K is given by (17.51 ) in Appendix. 

The presented model has the following heuristic justification: two disorders take place 
in the observed sequence They affect distributions by changing their parameters. 
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The disorders occur at two random times 6i and 62, 9i < ^2- They split the sequence of 
observations into segments, at most three ones. The first segment is described by {X^), the 
second one - for < n < ^2 - by {X}^). The third is given by (X^) and is observed 
when n > ^2- When the first disorder takes the place there is a "switch" from the initial 
distribution to the distribution with the conditional density /* with respect of the measure 
fi, where i = 1 ox i = 2, when Oi < 62 or 9i = 62, respectively. Next, if 61 < 62, at the 
random time 02 the distribution of observations becomes //^. We assume that the variables 
61 , 62 are unobservable directly. 

Let S denote the set of all stopping times with respect to the filtration {Tn), n = 
0, 1, . . . and T = {(r, cr) : r < a, T,a G S}. Two problems with three distributional 
segments are recalled to investigate them under weaker assumption that there are at most 
three homogeneous segments. 

2.1. Detection of change. Our aim is to stop the observed sequence between the two 
disorders.This can be interpreted as a strategy for protecting against a second failure when 
the first has already happened. The mathematical model of this is to control the probability 
P2^(t < 00, ^1 < T < ^2) by choosing the stopping time t* £ S for which 

(2.4) P^.(^i < T* < 62) = supP^.(r < oo,0i < r < 62). 

2.2. Disorders detection. Our aim is to indicate the moments of switching with given 
precision di, d2 (Problem D^^^j). We want to determine a pair of stopping times (r*, o"*) € 
T such that for every x € E 

(2.5) P,(|r* < -02| < ^2) - sup P,(|r - 0i| < di, |ct - ^2! < ^2). 

(T,<T)eT 



The problem has been considered in 12111 under natural simplification that there aie three 



segments of data (i.e. there is < < 02)- In the section |5]the problem Dqo is analyzed. 
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3. ON SOME A POSTERIORI PROCESSES 



The formulated problems are translated to the optimal stopping problems for some 
Markov processes. The important part of the reformulation process is choice of the statistics 
describing knowledge of the decision maker. The a posteriori probabilities of some events 



play the crucial role. Let us define the following a posteriori processes (cf. 1251], 12011 ). 



(3.1) Ui = P,(0i < n|/:„), 

(3.2) = p^^0^=0^>n\Tn)=PAOl = O2>n\J^mn), 

(3.3) Umn = Px{Ol = m,e2 > n\J^jnn), 

where Tm n = ioi m,n = 1,2, m < n, i = 1, 2. For recursive representation of 
(I3.1I )- (I3.3I ) we need the following functions: 



n^{x,y,a,f3,-f) = 1 



H(x,y,a,/3,7) 



[x,y,a,p,^) Uix,y,a,p,j) 



Il{x,y,a,(3,-f,6) 



il{x,y,a,(3,-f) 



il{x,y,a,^,j) 

where H(x, y, a, /3, 7) = (1 - a)pi/°(y) + [p2ia - /?) + qi{l -a- 7)]/i(2/) + fea + 
P2/3 + Qil]fx{y)- III the sequel we adopt the following denotations 

(3.4) a = (a,/3,7) 

(3.5) n„ = {ululul^). 



The basic formulae used in the transformation of the disorder problems to the stopping 
problems are given in the following 
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Lemma 3.1. For each x € E the following formulae, for m,n = 1,2, . . ., m < n, 
hold: 

(3.6) n^+i = ii^{Xn,Xn+i,ul^,ul,ul^) 

(3.7) Ul^, = U\Xn,Xr,+uK,K,K^) 

(3.8) n^+1 = n^^(x„,x„+i,n^, 11^,11;'^^) 

(3.9) 

with boundary condition Hg = vr, ng(x) = vrp, ng^(x) = vfp, and Hmm = (1 ~ 

^ rn — 1 

Proof. The cases (13. 6I ). (13.71 ) and ( 13.91 ). when < 6*1 < 02, have been proved in 



1 2511 and ll20ll . Let us assume < < ^2 and suppose that i?j G 1 < i < n + 1. Let us 



assume that Xq = x and denote = {a; : Xi{uj) G Bi, 1 < ? < n}. 

Ad. (13.61 ) For Ai = {u : Xi G Bj} G .Fj, 1 < i < n + 1 and Z)„+i G we have by 

properties of 5.„(x„) where £„ = (xq, . . . , x„) (see Lemma ItTI ) 

>n+l} 

Dn+l Dn+1 

(/r^^<^Hxi,„) + /r^i=^^(xi,.)) 
= J Q N ^ =r- — — 

x"+iB, '^nl^nj H(x„,X„+i, n„(x„)) 

= J (i-ni) ^ .iP.. 

Thus, taking into account (13.11) we have n^_|_i = 1 — P^; (^1 > n + 1 | J-'n+i) = 
1 — (1 — n^)pi/^^ (X„+i)H^^ {Xn,XnJri, ^n)- This proves the form of the formula 

KB. 



Ad. (13.71 ) Under the same denotations like in the proof of (13.61 ) we have using denotation 
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from Section im and the results of Lemma 1731 

= J — flx{dxi^n+l) 

n„(f„)) 

2 — 1 

r fen^(^0,n) +P2n2(fo,n) +gini^^(fo,n)]/x„(3;n+l) . 

= J — z iix[dxi^n+i) 



1—1 



-Dn+i H(x„, n„) 

Thus we get: 

= [(n^ - n2 + n2 + gini2] /j. jx„h.i)h-1(x„,x„+i, n„) 



which leads to the formula (I3.7I ). 
Ad. dXSl) By (IHl ) and the results of LemmaPO 

/ Px(^2 = ^1 > n + 1 I Tn+i)dP, = J I{9,=e,^n+i}dP, 

-Dn + l D„ + l 

fei=e2>niz: \ 

= j —=^ Hx{dXi^n+l) 

'S'n(^n)H(Xn,X„+i, n„,(^„)) 

Z — 1 * 

= j =i ^lx[,dXi^n+l) 

n+i^. H(x„,,x„+i, n„(x„)) 

i — 1 * 

Ai+i H(x„,x„+i, n„) 

which leads to: 



and it proves the formula (13.81) . 
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Ad. (13.91 ) Similarly, by the definition (13.31 ) and the results of Lemma 1731 we get 



/ Px{Oi = 771,02 > n + l\ :Fn+i)dPx = J \ei=m,e2>n+l}dPx 



D. 



n+1 



D. 



n + 1 



fMxidxi 



n+1; 



Xn+l) I N 

= j =^ fix{dxi^n+l, 

x"+lBi H(x„,X„+i, n„,(^n)) 

It leads to relation 



J Hm nP2/x„ (^n+l) 

z)„+i H(x„,x„+i, n„) 



nmn+l — P2nm n/x^C-'^n+l)!! (X^, , 11 , 



and it proves the formula ( 13.91 ). 

Further details concerning recursive formula for conditional probabilities can be found in 
Remark ITTT] in Appendix. 

Remark 3.1. Let us assume that the considered Markov processes have the finite state 
space and x„ = (xq, xi, . . . , xq = x are given. In this case the formula ( |j.9l ) follows 
from the Bayes formula: 



Px{Ol = j, 02 = k\Xn = Xn) = < 



P%U:=i fl_,{xs){Sn{xn)r' ifj > n, 



p%l{irj,fl_,{xs) 

X nr=, fk.M)iSn{xn))-^ ifj <n<k, 

. X Uu=k fL-ii^u){Sn{Xn))~^ if k < n, 



where pjj^ = P{9i = j, 62 = k) and Sn{-) is given by ( 17.51 ). 
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Lemma 3.2. For each x G E and each Borel function u : E — > the following 
equations are fulfilled: 

(3.10) - ni+i) I = (1 - ni)pi j2/)A.(dy), 

E 

(3.11) (ii(x„+i)(n,Vi - n^^i) I 

E 

(3.12) E, («(X„+i)n2^i) I = [gsni +p2n2 + q,Ii'^]ju{y)fl^{y)ii{dy), 

E 

(3.13) E, (ii(x„+i)ni2^i) I Tn) - pini2 (y)M(rfy) 

E 

(3.14) E,(n(X„+i)|J-,) = / n(y)H(X„,y, n^„)^(dy). 

E 

Proof. The relations (I3.10I )- (I3.13I ) are consequence of suitable division of Q de- 
fined by (01, 62) and properties established in Lemma 1731 Let us prove the equation (13.121 ). 
To this end let us define (T-field = ^2, -^O) •••j -'^n)- Notice that jr„ C J^„.Wehave: 

^^{u{Xn+l)K+l I 

Sjn+1} I -^n) I -Fn) 

= E^(I|e^^„+i}E,(n(X„+i) I ^n) I = / n(y)/ijy)/i(dy)P,(02 <n+l\Tn 

E 



E 

We used the properties of conditional expectation and point 5 of Lemma 1731 Similar trans- 
formations give us equations (I3.10I ). (13.131 ) and (13.111 ) when the points 1 and 2, the point 4 
and the point 1 of Lemma 1731 respectively. From (I3.10I )- (I3.12I ) we get (13.141 ). The proof of 
the lemma is complete. 
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4. DETECTION OF NEW HOMOGENEOUS SEGMENT 

4.1. Equivalent optimal stopping problem. For Xq = x let us define: Z„ = Px (^i < 
n < ^2 I ^n) for n = 0, 1, 2, . . .. We have 

(4.1) Zn = Px(^l < n < ^2 I -^n) = - n2 

Yn = esssupj^gT-^ ^^n}Px(6'i < T < 6*2 | -Fn) for n = 0, 1, 2, . . . and 

(4.2) To = mf{n > : Z„ = y„} 

Notice that, if Z^o = 0, then Zr = Pxl^'i < r < 6*2 | J^r) for r € T. Since JT^ C jr^ 
(when n < r) we have 

y„ = esssupEx(Z^ I JFn). 

Lemma 4. 1 . The stopping time tq defined by the formula ([4.2\l is the solution of the 
problem d2.4D . 

Proof. From the theorems presented in yD it is enough to show that lim Zn = 0. 

n— >oo 

For all natural numbers n,k, where n > /c for each x G E we have: 

Zn = Ex{I[g^<:n<e2} I ^n) < -E'x (sup Ijg^ <gj <6»2} I ^n) 

From Levy's theorem lim sup„_,oo -^n < E^{supj^kl{g^^j^g^} \ Too) where Too = 
i that: lim supl{ei<gj<e2} = 

gence theorem we get 



a (UJ^i ^n)- It is true that: lim supl|e^<,<g2| = a.s. and by the dominated conver- 



lim £;2;(sup W<c,<92| I -^oo) = a.s. 
what ends the proof of the lemma. 



The reduction of the disorder problem to optimal stopping of Markov sequence is the 
consequence of the following lemma. 
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Lemma 4.2. System X'-^ = {X^}, where X^ = {Xn-i, Xn,Ill^,U'^,Ul^) forms a 
family of random Markov functions. 

Proof. Define a function: 
(4.3) (f{xi,X2,d iz) = {x2,z,U^{x2,z,d),U^{x2,z,a),U^'^{x2,z,d)) 
Observe that 

■^n = ^{Xn-2,Xn-l, ^n-l',Xn) = ip{X^_i; Xn) 

Hence can be interpreted as the function of the previous state X!l^_^ and the random 
variable Moreover, applying (13.141 ). we get that the conditional distribution of X„ given 



fj-field Tn~i depends only on X!f^__^. According to 11191] (pp. 102-103) system is a family 
of random Markov functions. 

This fact implies that we can reduce the initial problem (12.41 ) to the optimal stopping of the 
five-dimensional process (X„„i, H^, 11^, 11^^) with the reward 

(4.4) h{xi,X2,a) = a — (3 

The reward function results from the equation (14.11 ). Thanks to Lemma |4^ we construct the 



solution using standard tools of optimal stopping theory (cf 111911 ), as we do below. 

Let us define two operators for any Borel function t; : x [0, 1]'^ — > [0, 1] and the 

set D = {uj: Xn-i = y,Xn = z, = a, = /3, = 7}: 

Ta:V{y,Z,d) = Ea:{v{Xn,Xn+l,'nn+l) \ D) 

Qxviy,z,d) = m.ayi{v{y,z,d),T:,v{y,z,a)} 

From the well known theorems of optimal stopping theory (see I19I1 ). we infer that the 
solution of the problem (12.41 ) is the Markov time tq: 



(4.5) To* = inf{n > : n„+i) > n„+i)}, 
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where: 

h*{y,z,a)= lim Q'^h{y,z,a). 

k—foo 

Of course 

To obtain a clearer formula for Tq and the solution of the problem (12.41 ). we formulate (cf 
(1331) and (134)) ): 

Theorem 4. 1 . (a) The solution (14.51 ) o/f/ze optimal stopping problem for the stochas- 
tic system defined in Lemma \4/2\ with payoff fiinction l \4.4i is given by: 

(4.6) To* = inf{n > : n„+i) £ B*}. 
Set B* is of the form: 

B* = {{y,z,d) : {a - P) > [1 - a--f) \pi J R*{y,u,Ui{y,u,a))fy{u)i^{du) 

E 

+ QiJ S*{y,u,Ui{y,u,d))fy{u)i^i{du)] 

E 

+ {a-l3)p2j S*{y,u,Ui{y,u,a))fy{u)fi{du)\ , 

E J 

where R*{y,z,a) = liuik-^oo R'^iu, z, a), S*{y,z,d) = liuik^oo S''{y, z, a). The 
functions R^ and are defined recursively: R^{y,z,d) = 0, S^{y,z,d) = 1 and 

(4.7) i2'^+i(y,z,a) = (l-%,(?/,z,a)) (pi J R^iy,u,Uiiy,u,a))fll{u)fiidu) 

\ IE 

+QiJ S^{y,u, Ui{y,u,d))fy{u)fi{du) , 

E / 

(4.8) 5^+i(y,z,a) = I7^, (y, z, 5) + (1 - % (y, ^, «)) 

XP2 J S^{y,u, Ui{y,u,d))fy{u)p,{du), 

E 
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where the set TZ^ is: 

(4.9) 7^fc = : h{y,z,a) > T,Q^-i/i(y, z, a)} 

= {(y, 5;, a) : (a - /?) > (1 - a - 7) 

X pi J R''{y,u,Ili{y,u,a))fll{u)fi{du) 

E 

+ 91/ S'^{y,u, Ui{y,u,d))fy{u)n{du) 

E 

+ (a- (3)p2j S\y,u,Tl\y,u,a))f^{u)^i{du) 

E 

(b) r/je optimal value for f l2.4D g/ven ^73^ the formula 

V{x) = ma,yi{p2Tt p,Vq{x)} 



where 



Vq{x) = np 



pi J R*{x,u, Ili{x,u,Tr, p-K, pTr))f^{u)p{du) 



+qi J S*{x,u,Ui{x,u,Tr,pTr,p{l -Tr)))f^{u)p{du) 



+ 7rpp2 J S*{x,u,Ui{x,u,'K,p7r,p{l - -K)))f^{u)p{du) 

E 

and T* = 0^p27tp;?Vo{x)} + Tohp2^p<Vo{x)}- 
Proof. Pait (a) results from Lemma 1X2] - the problem reduces to the optimal stop- 
ping of the Markov process X„, 11^, 11^, 11^^) with the payoff function h{y, z, a) = 
a— p. Given (13.111 ) with the function u equal to unity we get on D = {uj : Xn-i = y, = 

T^h{y, z, a) = (n^+i - Ul^^ \ Tn) \d 



((1 - - n^2)gi + {Yi\ - n2)p2) / 

E 

{I - a- 7)(?i + (a - /3)p2- 
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From the definition of and 5^ it is clear that 

h{y, z,d) = a — (3 = {1 — a — j)R^{y, z, a) + (a — (3)S^{y, z, a) 

Also TZi = {{y, z, a) : h{y, z, a) > Txh{y, z, a)}. From the definition of Q^,- and the facts 
above we obtain 

Clxh{y,z,a) = {I - a - -f)R'^{y, z,a) + {a - ^)S'^{y, z,a), 

where R^{y,z,a) = qi{l - l7e,(y,z,a)) and S'^{y,z,a) = p2 + {I - p2)l'R.^{y, z,a)). 
Suppose the following induction hypothesis holds 

Qt^h{y, z,a) = {l-a- j)R''{y, z, a) + {a - P)S^{y, z, a), 

where R^ and are given by equations (14.71 ). (14.8b . respectively. We will show 

Q^/i(y, z,a) = {l-a- j)R''+Hy, z, a) + (a - f3)S^^\y, z, a). 

From the induction assumption and the equations (13.101) . (13.131 ) and (13.111 ) we obtain: 

(4.10) T^Q^-i/i(2/, z, a) = T,(l -a- ^)R\y, z, a) 

+ T:^{a-P)S\y,z,a) 
= (l-a-7)pi / R''{y,u,Ili{y,u,a))fy{u)fi{du) 

E 

+ [(1 - a - 7)gi + (a - /3)p2] J S^{y,u, ni{y,u,a))fy{u)i^{du) 



(l-a-7) 



PijR''{y,u, Ui{y,u,a))fy{u)ij{du) 



Notice that 



+91 / S'^iy, u, Uiiy, u, a)) fy (u) n{du) 

E 

+(a-/3)p2 / S''{y,u,Ili{y,u,d))fy{u)^i{du). 

E 

(1 _ a - ^)R>'+\y, z, a) + (a - ^)S''+\y, z, a) 
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is equal a — (3 = h{y, z, a) = Q^h{y, z, a) for {y, z, a) € TZk and, taking into account 
(14.101) . it is equal TxQx~^h{y, z, a) = Q^h{y, z, a) for {y, z, a) ^ TZk, where TZk is given 
by (14.91 ). Finally we get 



Q'My, z,a) = {l-a- j)R'+\y, z, a) + {a - l3)S'+\y, z, a). 
This proves (14.71 ) and (14.81 ). U sing the monotone convergence theorem and the theorems of 



optimal stopping theory (see 11911 ') we conclude that the optimal stopping time Tq is given 

bym. ^ 

Proof. Part (b). First, notice that U\, and 11^ ai-e given by (|331)-(|3^ and the 
boundary condition formulated in Lemma [3TT] Under the assumption t* < oo a.s. we get: 

P^(t* < oo, 6*1 < t* < 6*2) = sup BZr 

T 

= 'Emax{h{x,Xi,Ui),T^h*{x,Xi,Tii)} = E lim Q^/i(x, Xi, Hi) 



k^oo 



E 



[i-n\- ul'')R*{x, Xi, Hi) + {u\ - ul)s*{x, Xi, n 



TTppi J R*{x,u, Ili{x,u,7r, piT, p7T))f!^{u)fi{du) 



E 



+ {irpqi + 7rpp2) J S*{x,u, Ui{x,u,Tr, pir, pTr))f^{u)p{du). 

E 

We used Lemma lTll here and simple calculations for Iff and 11^. This ends the proof. 

4.2. Remarks. It is notable that the solution of formulated problem depends only on 
two-dimensional vector of posterior processes because 11^^ = p{l — IlJ^). The obtained 
formulae are very general and for this reason - quite complicated. We simplify the model 
by assuming that P{9i > 0) = 1 and P(02 > (^i) = 1- However, it seems that some further 
simplifications can be made in special cases. Further research should be carried out in this 
direction. From a practical point of view, computer algorithms are necessary to construct 
B* - the set in which it is optimally to stop our observable sequence. 
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5. IMMEDIATE DETECTION OF THE FIRST AND THE SECOND DISORDER 

5.1. Equivalent double optimal stopping problem. Let us consider the problem Dqo 
formulated in (12.5b . A compound stopping variable is a pair (r, a) of stopping times such 
that < T < cr a.e.. The aim is to find a compund stopping variable (r*, a*) such that 

(5.1) P^((0i,02) = (T*,a*))= sup P,((0i,02) = (T,fT)). 

Denote Tm = {(''", f) € T : r > m}, 7^„ = {(r, a) G T : r = m,cr > n} and 
Sm = {t G 5 : T > m}. Let us denote J-'mn = -^n> m, n G N, m < n. We define 
two-parameter stochastic sequence (^{x) = {^mn, m, n G N, m < n, x G E}, where 

iran = ^x{Ol =m,02 = n\Trnn)- 

We can consider for every x G E, m, n G N, m < n, the optimal stopping problem of ^{x) 
on = {(r, cr) G T^n : t < a}. A compound stopping variable (r*, o"*) is said to be 
optimal in (or T^^) if 

(5.2) ^xiT*a* = sup E^^^^ 
(or 'F:^ir*a* = sup(^^^-)g^+^ E^.^^<^). Let us define 

(5.3) 

'?mn — ess sup 'Eix{^£,Ta\-F rnn)- 

If we put ^moo = 0, then 

rjmn = ess sup PxiOl =T,e2 = Cr\J^mn)- 

From the theory of optimal stopping for double indexed processes (cf. IsIi. IibI] ) the sequence 
r]mn satisfies 

'Hinn — ^'^^{Cmm^iVmn+llJ' mn)}- 

Moreover, if cr^ = inf{n > m : rjmn = ^mn}, then (m, cj*) is optimal in and 
rjmn = ^x{^ma*\^mn) a.c.. The casc whcu there are no segment with the distribution 
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/^(y) appears with probability p. It will be taken into account. Define 

?7mn = max{^m„, E(7?m „+i|-Fmn)}, for n> m. 

if a*^ = inf{n > m : ?}„„ = fmn}, then (m, a^) is optimal in %nn and r)mm = 
^x{ima^ \^mm) a.c.. For further consideration denote 

(5 .4) r]m = (??,nm+ 1 1 -^m ) • 

Lemma 5.1. The stopping time crj^ is optimal for every stopping problem d5. JD . 

Proof. It suffices to prove lim„^oo imn = (cf. yfl). We have for m,n,k G N, 
n ^ k > m and every x € E 

Ea;(I{5)j=m_e2=„}|:Fm„) = ^mn{x) < E^; (sup I{0^=m,e2=j} l-^m) > 

where 1^ is the characteristic function of the set A. By Levy's theorem 

n-*oo j^k 

where JT^ = J^noo = o-(U-^=i ^n)- We have lim suplig^=rn,e2=j} = a.e. and by domi- 



nated convergence theorem 



lim E^(supl{ei=m,02=i}l-^oo) = 0. 

What is left is to consider the optimal stopping problem for (r/mn)m=o "^-m '^^ 
{%nn)'^f^^n=m- Let US define 

(5.5) Vm = esssupEa;(?7^|:r^). 

Then Vm = max{7]m, Ex{Vrn+i\J^m)} a.e. and we define r* = inf{A; ^ n : Vk = 
Lemma 5.2. The strategy Tq is the optimal first stop. 
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Proof. To show that Tq is the optimal first stop strategy we prove that P^; (tq < 
oo) = 1. To this end, we argue in the usual manner i.e. we show lim-m^oo flm = 0. 
We have 

rim = '^x{ima^\^m) ='^x(^x(J{ex=mfi2=<T'f^}\^rnal^)\^m) 
= ^x{'^{ei=mfi2=a*-^}\^rn) <^x{s'^'9'^{ei=jfi2=cj''}\^m)- 

Similarly as in proof of Lemma ISTT] we have got 

m— >cxD j^k ^ 

Since linifc^oo supj^^ l{ei=k,e2=a*} < limsupfc^oo \ei=k} = 0, it follows that 

lim r]mix) < lim E^{suplfg *y\J^oo) =0. 

Lemmas [5.11 and [5^ describe the method of solving the "disorder problem" formulated 
in Section |2] (see (15.11 )). 

5.2. Solution of the equivalent double stopping problem. For the sake of simplicity we 
shall confine ourselves to the case di = d2 = 0. It will be easily seen how to generalize 
the solution of the problem to solve D^^j^^ for c?i > or (i2 > 0. First of all we construct 
multidimensional Markov chains such that and rjm will be the functions of their states. 
By consideration of Section |3] concerning a posteriori processes we get ^oo = t^P and for 
m < n 



=3 



TTp 



Sn{xo,Xi, . . . , Xnj 



P2 fk-S^n) 
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and for n = m, by Lemma 1731 

(5-6) (.^rn = Px{9l = m, 02 = m\Tjnrn) = p— 'In" ' ) ~ ^m)- 



qi fxrn-A^-" 



We can observe that {Xn, Xn+i, n„+i,nmn+i) for n = m + 1, m + 2, . . . is a func- 
tion of {Xn^i, Xn, Iln,^mn) and Xn+i- Bcsidcs, the conditional distribution of Xn+i 
given Tn (cf. (13.14b ) depends on Xn, n^(x) and n^(a;) only. These facts imply that 
{{Xn, Xn+uJ^n+i,'n^mn+i)}'^=rn+i form a homogcncous Markov process (see Chap- 
ter 2.15 of il9fl ). This allows us to reduce the problem (15.31 ) for each m to the optimal stop- 
ping problem of the Markov process Zm{x) = n„, Ilmn), m,n G N, m < 
n, X G E} with the reward function u, a, (5) = ji^y 

Lemma 5.3. A solution of the optimal stopping problem f l5.JD for m = 1,2,... has a 
form 

(5.7) < = inf{n > m : f-\ > 

where R*{t) = p2 /jg ^*(*) ■5)//(s)/i((is). The function r* = lim^^oo '"n. where ro(t, u) = 

(5.8) r„+i(t,u) = max{ * p2 J rn{u, s)fl{s)fi{ds)}. 

h v^J E 

So r*{t, u) satisfies the equation 

(5.9) r*{t,u) = max{^^,p2 / r*{u,s)fl{s)p{ds)}. 
The value of the problem 



(5.10) 77^ = E,(r?^™+i|^„) = - ,n , (1 - n^)i?*(X^-i, X, 



(5.11) i?*(t,u) =max{p^,^(l-p)i?»}. 
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Proof. For any Borel function n : E x E x [0, l]'^ ^ [0, 1] and D = {uj : Xn-i = 
t, Xn = u, Unix) = a, 11^(2;) = /3, = 7, Ilmn{x) = S} let us define two operators 



On the bases of the well-known theorem from the theory of optimal stopping (see lll9l] . lllSh ') 
we conclude that the solution of (15.31 ) is a Markov time 

o-;^ = inf{n > m : /i(X„_i,X„, n.„,nmn) = /i*(X„_i,X„, Un{x),Umn)}, 

where h* = linifc^oo QxHi^u,d,5). By dSUl and (I37T4]) on Z) = : = t,Xn = 

u,Ul = a,Ul = /3,n^2 = 7,nmn = <J} we have 



Txu{t,u,d, 6) 



Ex{u{Xn,Xn+l, Un+lix),Um.n+l{x))\D) 



and 



Qxu{t, u, a, 5) = max{n(t, u, a, 6), Txu(t, u, a, 5)}. 




Txh{t,u,d, 6) 



IXT41 




and 



(5.12) 



Clxh(t,u,d,6) = — (5max{ 

P2 



P2}- 



Let us define ro(t, u) = 1 and 



rn+i{t,u) = max{ 



,P2 J rn{u,s)f^{s)fi{ds)}. 



E 



We show that 



(5.13) 



Q h{t, u, a, 6) = — Sr£{t, u) 

P2 
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P2 

'-Sp2 J rkiu,s)f^{s)n{ds). 

P2 E 



for ^ = 1, 2, .... We have by (I57T2] ) that Q^h = ^^ri. Let us assume (l57T3l) for £ <k.By 
(13.141) on = {tj : X„_i = t,Xn = u, 11^ = a, 11^ = /3, II^^ = 7, Ilmn = S} we have 
got 

= ^2 

It is easy to show (see H) that 

Q^;+^/i = max{/i, T^Q^/i}, for /c = 1, 2, . . .. 
Hence we have got Q^+^/i = ^Jr^+i and (15.131 ) is proved for £ = 1,2, This gives 

(5.14) h*it,u,a,6) = —6 lim rk{t,u) = —6r*{t,u) 
and 

Vmn — 6SS sup 'Eix{^t,(t\-^ m.n) — ^ {Xn~li Xji, ^m^mn)- 

We have by (l5l4l) and (13^91) 

T^h*(t,u,a,S) = —5p2 f r*(u,s)fl(s)fi(ds) = ^5R*(u) 

and a*^ has form (|5^ . By (|5H) . (1531 ) and (13.141) we obtain 

(5.15) = max{^^„,E(?7„™+i 

Remark 5.1. Based on the results of Lemma 15. J I properties of the a posteriori 
process linm we have that the expected value of success for the second stop when the ob- 
server stops immediately at n = is Trp and when at least one observation has been made 
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E(,?i|^o) = f^E((l -ni)|§i|ii^(x,XO|^o) = f^{l-^)pi^^fl{u)Rl{x,u)^i{du). 
As a consequence we have optimal second moment 

if Tip > gi(l - vr)/ f]^{u)R'p{x,u)n{du), 

E 

(Jq Otherwise. 

By Lemmas 15.31 and 13.11 (the formula (13.91 )) the optimal stopping problem (15.51 ) has been 
transformed to the optimal stopping problem for the homogeneous Markov process 

with the reward function 



(5.16) /(t,^,a) = ^^(l-a)iZ^(t,n). 



PI fKn) 

Theorem 5.1. A solution of the optimal stopping problem (15.51 ) for n = 1,2, . . . has 
a form 

(5.17) T* = mf{k > n : {Xk-i,Xk,Tlk,) e B*} 

where B* = {{t,u,a.) : jr^B*p{t,u) > pi J^v*{u, s)f^{s)fj,{ds)}. The function 
v*{t, u) = lim^^oo Vn{t, u), where vo{t, u) = R*p{t, u), 

(5.18) Vn+i{t,u) = max{^^^i2^(t,n),pi / Vn{u,s)fl{s)p{ds)]. 
So v*{t, u) satisfies the equation 

(5.19) r;*(t,n) =max{^|^i?*(t,u),pi / v*{u,s)fl{s)ii{ds)}. 

h l^J E 

The value of the problem Vn = v*{Xn-i,Xn)- 



Proof. For any Borel function u : E x E x [0, l]'^ ^ [0, 1] and D = {uo : Xn 
t, Xn = u, n^(x) = a, 11^(2;) = /3, = 7} let us define two operators 



-1 



,11,0:) — ^x{'u{Xn, Xn+l, ^n+l 

)\D) 
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and Qxu{t, u, a) = max{ii(t, u, a),Txu{t, u, a)}. Similarly as in the proof of Lemma l5.3l 
we conclude that the solution of (15.51) is a Markov time 



= inf{n > m : n„) = n„)}, 

where /* = limfc_>oo Q^/(t,n, q). By (13.141 ) and (15.161 ) on D = {uj : = t,Xn = 

u, nl = a,Ul = /?, = 7} we have 

Txf{t,u,a) = E.(^(l-n^^,) {f j^"+^| fi-(X„,X.+i)|ZJ) 

= — (l-a)pi j — --H{u,s,a,f3)R {u,s)fi{ds) 

Pi E H{u,s,a,p) ^ 

qi 



(1 - a)pi / R*p{u,s)fl{s)^i{ds 

P^ ¥. 



and 



(5.20) Q,/(t,t/,c5) = ^(l-a)max{^i?*(t,n),pi/i?*(n,s)/^(sMds)} 

= — avi{t,u). 
Pi 

Let us define u) = max{^^0jR*{t,u),pi J^R*p{u, s)f^{s)fj,{ds) and 

f^(u) 

Vn+i{t,u) = max{-^— i?*(t,n),pi / s)/°(s)/x((is)}. 

Jt \'^) E 

We show that 

(5.21) Qi./(t, u, a) = ^(1 - a)viit, u) 

Pi 

fori= 1,2,.... We have by (15.201 ) that Qxf{t, u, a) = ^{l — a)vi{t, u) and let us assume 
(I52T]) for ^ < fc. By (I3l4b on D = {cj : = i, X„ = H;^ = a, = /3, = 7} 

we have got 

T,Q^/(t,n,a) = ¥,x{-{l-Tll+^)vk{Xn,Xn+i)\D) 

Pi 

= — (l-a)pi / Vk{u,s)fll{s)fx{ds). 
Pi E 
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Hence we have got Q^"*"^/ = ^(1 — a)ffc+i and (15.211) is proved for £ = 1,2,.... This 



p 

gives 



f*{t, n, a) = — (1 — q) lim Vk{t^ u) = —av*{t, u) 

Pl k^oo pi 



and 



We have 



Vm = -{l-Ui)v*{Xm-l,X„,). 
Pl 

TJ*{t,u,d) = ^(l-a)pi / v*{u,s)f^{s)fi{ds). 

E 

Defined* = a) : ^^R*{t,u) > pi Jj^, s)/°(s)/x(ds)} then r* for n > 1 has 

a form (15.171 ). The value of the problem (15.21) . (15.51 ) and (12.51) is equal 

fo(a;) = max{7r,E^(yi|J"o)} = max{7r, — (1 - 7r)pi [ v* {u, s)f^{s)ix{ds)} 

and 

' if 7r> qi{l-Tr) J v*{u,s)f^{s)fi{ds), 



Tq otherwise. 



•5< 



Based on Lemmas 15.31 and [5TT] the solution of the problem Dqo can be formulated as 
follows. 

Theorem 5.2. A compound stopping time (r*, o"*, ), where is given by 0.71 ) and 
T* = Tq is given by ( 15. 1 7\l . is the solution of the problem Dqq. The value of the problem 

V^{t* <a* < oo,^i = r*,02 = <.) =max{7r,gi(l-7r) / v* {u, s)fll{s)fi{ds)}. 

E 

Remark 5.2. The problem can be extended to optimal detection of more than two 
successive disorders. The distribution of 6i, 62 may be more general. The general a priori 
distributions of disorder moments leads to more complicated formulae, since the corre- 
sponding Markov chains are not homogeneous. 
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6. FINAL REMARKS 

It is notable that the final optimal solutions turns out to have an unexpectedly simple 
form. It seems that some further simplifications can be made in special cases. From a prac- 
tical point of view, computer algorithms are necessary to construct B* - the set in which 
we stop our observable sequence. Since we always refer to the transitions densities it is still 
open problem of switching between the independent Markov sequences. 

7. APPENDICES 
APPENDIX 1 — USEFUL RELATIONS 

7.1. Distributions of disordered samples. Let us introduce the n-dimensional distribu- 
tion for various configuration of disorders. 

n j~l n 

(7.1) = vfp E n fLA^s) n fLM)} 

j=l s=l t=j 

n—1 n j— 1 fe— 1 n 

E E {p^i~\ipr'~\2 n fi.A^s) n fk.M) n 

j=l k=j+l s=l t=j u=k 

n 

s=l 

n . j-l n 

(7.2) /f^^"<^n^i,n) = ^pT.{pi\ipr' n fi-.i^s) n fk.M)} 

3=1 s=l t=j 

n j — 1 n 

+ vrp E{pr'92 n fLA^s) n fLs^t)} 
j=i s=i t=j 

n 

(7.3) /f^=^^>"(xi,„) = pTTp^lUfLA^s) 

s=l 
n 

(7.4) /r'^<^^(xi,„) = pTtPiUfl-A^s). 

s=l 

Let us define the sequence of functions Sn ■ x^LiE ^ as follows: So{xq) = 1 and for 
n > 1 
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Lemma 7.1. For n > the function Sn{xi.n) follows recursion 

(7.6) 5'„+i(xi,„+i) = H(a;„,a;„+i, n„)S'„(xi,„) 

(7.7) H(x,y,a,/3,7) = (1 - + b2(a - /3) + - a - 7)]/.'(y) 

+ [q2a + P2P + qii]fl{y). 

Proof. Let us assume < (?i < and suppose that e B, 1 < i < + 1 and let us 
assume that = x and denote £>„ = {w : Xi{ui) G 5^, 1 < i < 71}. For Ai = {uj : Xi ^ Bi} g 
J^^i, 1 < i < ?i + 1 we have by properties of the density function Sn{x) with respect to the measure 

/ dPx = / S'„+i(x„+i)/i(dfi^„+i) 



i = l 



= / / /(a;„+i|f„)/x((ia;„+i)S'„(fo,n)Ai(rf^i,n) 

= / P,(A„+i|Xi,„)dP, = / P,(A„+i|.F„)dP, = / U„+idP, 

D-n -D„ -D„ 

Now we split the conditional probability of An+i into the following parts 

(7.8) P.(X„+i e A„+i I Tn) = Px(n < 01 < e2,^„+i e 

(7.9) + P^{01 < n < 02, Xn+l G A„+i I Tn) 

(7.10) + P,(?i<0i==02,^„+ieA„+i |.F„) 

(7.11) + P,(0i <02 <n,X„+i G A„+i |.F„) 
In ( 17.8b we have: 

=11+1} + I{0i>n+1 

D„ D„ 

J {f:<''<'Hxi,n){pifU^n+l) + qifljXn+l))Kdxi,n+l) 

= I / bl/l(x„+i)+gi/ijx„+i))/i(d.T„+i))A*(dfl,„) 
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In ( |7.9l l we get by similar arguments as for (17.81 ) 

P,{9i < n < 02 , Xn+i e A„+i I T„) 

= P,{9i < n< 02,^2 =n + l,X„+i G A„+i I 

+P:,(6'i < n < 6*2, 6I2 7^ n + 1, X^+i e A„+i | J^„) 

= {PAOl < n I Tn) - Px(^2 < n I ^„)) 
x[g2P|„(A„+i) +p2Px„(^n+l)] 

In ( 17.1 lb this part has the form: 

Pa; (^2 < n,Xn+i G An+l ri+1 ) 

In (17.10b the conditional probability is equal to 

Px{Oi =02>n , Xn+i e A„+i I J^„) 

= P:,(6li = 6*2 > n,e'2 =n+l,Xn+l e An+l I 

+P^(e'i = 6I2 > n, 6*2 ^ n + 1, X„+i e An+i \ Tn) 
= P,(0i ^d2>n I .F„)[<ziP|jA„+i) +piP|^.JA„+i) 

These formula lead to 

f{Xn+i\Xi_rL) = H(x„, x„+i, 11;',, n^, n^^). 

which proves the lemma. 



7.2. Conditional probability of various events defined by disorder moments. According 
to definition of 11;',, 11^,, II}^ we get 

Lemma 7.2. For the model discribed in Section^the following formulae are valid: 

1. p,(02 >ei>n\Tn) = i- ni - = /if^^Jl^; 

2. p,((?2 = 01 > = = zli^i^ipi); 

3. p,(0i <n<02|^„) = ni -n2; 



4. P,(02>ei>n|^„) = l-ni 
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Proof. 

1 . We have 

(7.12) n = {uj : n< 9i < 62} U {uj : 9i n < 62} 

U {cj : 6*1 < 6*2 < n} U {cj : 6*1 = 6*2 > n}. 

Hence 1 = P^^{iu : n < 9^ < 021^,,) + (n,\ - ) + + n,\2 and 

F^iu: n<9i< 92\Tn) = 1 - - n^^. 

Let i?,: G Z?, 1 < i < n, Xq = x and denote £)„ = {w : Xi{uj) € Bj, 1 < i < 71}. For 
= {cj : e e !Fi, 1 < « < n and L)„ G Tn we have 

/ I{e,>ei>„}dP. = / Pxi02 >0i> n\T„)dP, = / P,(02 > > ^^|^r^)^iP. 

D„ D„ D„ 

= / /;'<«^<«HXi.„)(5„(X„))-idP,. 

Thus P,(02 > ^1 > n|J^„) = pTTVlKi=l fx^^M^){Sn{X,,))-\ 

2. The second formula can be obtained by similar argument. 

3. Let 9i < 6*2. Since : ^2 < n} C {tJ : 6*1 < n} it follows that P:r({i^ : 6*1 < n < 
OnWu) = Px({w : 01 < n} \ {c^ : 02 < n}\T^) = - H^. 

These end the proof of the lemma. 

Remark 7.1. Let Bi e B, l ^ i ^ n +l, Xo = x and denote Ai = {w : Xi(a;) G Sj, 1 < 
i < For ylj = {cj : G G .F,, 1 < i < n ant/ D„ G we have 

J l[e,>n}dP^ - / P,(0i > n|.F„)dP, ^ / P,(0i > n|X„)dP, 

n 

= P49i> n,Dn)^ J Pi U fl.M)f'idxi,^) 

n 
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Thus P,(0i > n|^„) = nr=i /i'.-i {Xi){Sn{Xr,))-\ Moreover 

and Sn+i{Xn+i) = H(X„,X„+i, n,\)S'„(X„). Hence 

^1 pi/°jX„+i)(l-n,\) 
J-J-n-l-l — J- =; • 

H(x„,x„_|_i, n„) 

7.3. Some recursive formulae. In derivation of the formulae in Theorem l3. li the form of the 
distribution of some random vectors is taken into account. 

Lemma 7.3. For the model discribed in Section^the following formulae are valid: 

1. P,((?2 = 01 > n + l\Tr,) = PiK^ = Pip{l - K); 

2. p,((?2 >ei>n + = - Hi - ni2); 

4. PAOl <n + i<02\T^) = <zi(i - n,! - ni2) +p2(ni - n^),- 

5. PAd2 <n+ 1|.F„) = q2ni + P2n2 + q.Ull 

6. P:r(6'l =m,02> n+ l\Tn) = P2^in n- 

Proof. 

1. On the set D ^ {lj : Xq = x, Xi e Ai, X2 e A2, . . . , Xn & An} S we have 
/l{e,=ei>„+i}dP^ = P.(-D)Px(02 =01 > n + 

00 ri 

= P7f yr^gi / n /x,_i(a;j)/^(rf^i,«) 

j=n+2 xJ'^jAii=l 
n 

= PiP,p)P,(02 = 01 > = pi / I{(,,=e,>„}dP,. 

n 

By ( I3.2I 1 and the definition of the conditional probability this implies Px{S2 = 0i > + l|^n) = 
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PiTll"^. Next, 

/l{ei>„}rfP. = VAD)Vx[Oi>n\D) 

D 

oo n 

= -P,{D)P,,{02 =ei> n\D) = - j I{e,=ei>„}dPx. 

P P D 

These prove the part 1 of the lemma. 

2. Similarly as above we get 

/l{e2>ei>«+i}rfPx = P{D)P,,{e2> 9i>n + l\D) 

D 

n 

= PlPT^Pl J n fxi^ii->^t)p{dxi,n) 

= piP{D)P,{02 >e^>n\D)=pij I{9,>9i>„}dPx 

D 

By point 2 of Lemma l772l we get the formula 2 of the lemma. 

3. It is obvious by assumption 6i < 02- 

4. On the set D we have 

/l{ei^„+i<e.}rfPx = P{D)P,{9i^n + l<e2\D) 

D 

n 

= 7fp5'(?i(l -p) / U fL-li^s)f^idxi^n) 

n j - 1 n 

+P2EP(^ : ^1 = / n fi^A^s) n /i..,(x.)M(dfi,„) 

^'^PgiP(Z?)P,(02 > ^1 > +p2Pp)P.(0i < < 02|i?) 

= 9l / I{e2>6li>n}'^PK +P2 / I{ei^n<6l2}^^Px- 
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5. If we substitute n by n + 1 in ( 17.12b than we obtain 

-P,(n + 1< 01 < 02|^«) - P.(0i < n + 1< 02|^„) 

= 1 - pini2 - pi(i - ni - ni2) - ^^(i - - n^^) 

6. We have 

/ I{ei=rn,e2>n+l}dPx = P,(i?)P.(01 = m, 02 > ^ + 1|Z?) 
D 

OO rn n 

■m 71 

^?=l^i 1=1 j=m+l 

= P2Pa;(-D)Px(01 ^ 171,02 > n\D) = P2 f l{ei=rn,e2>n}dP x ■ 

D 

By (13.3b and the definition of conditional probability this implies Pa;(02 — m,9i > n + l\Tn) = 
P2^n m- These prove the part 6 of the lemma. _ 
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